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period in one language and then translat^e briefly to students who do 
not understand; and (7) set language development as the main goal of 
the class for the school year. (BW) 
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SAME BILINGUAL PRESCHOOL PROJECT, DIFFERENT GAINS; WHAT HAPPENED? 



INTRODUCTION 

During ^Che 1982-83 school year the Austin Independent School District (AISD) 
implemented a sucessful bilingual preschool project. The data collected 
for tl^e evaluation of this^project allowed us to compare classes within 
the Project to identify factors that might lead to greater academic 
achievement gains and language development. 

. , : , ' ^ . ' ' ' ' 

The AI$D Bilingual Preschool Project consisted of six classes in. five, 
elementary neighborhood schools. The Project was planned to address 
the ne,eds of the four-year olds who were classified as students with 
limited English proficieucy (LEP). The maiu objective of the .Project 
was tljie following: / 

o Project students will attain a higher level of skill 
in language and concept development as measured by 
the Peabody Picture Vocabulary Test- Revised version (PPVT-R) . 

Other educational goals of the Project were to teach children to be 
independent, to learn appropriate school behavior, and to learn to 
interact with peers and adults. - » 



WHAT WERE THE CHARACTERISTICS OF THE PROJECT? 

The Project had the following characteristics: " V 

o Teachers were certified in Early Childhood Education and 
were bilingual in English and Spanish. 

' ^ >. 

6 Teachers used two languages for ' instruction, English and 
Spanish. 

o English was . the predominant language for instruction. 

However, basic concepts and ideas were taught in the primary 
language of the student. * ^ , i 

o It was planned that at .least 50% of the regular school day 

(390 minutes) would be dedicated to instruction. 

■ • ^ 

\ o Each preschool class included three students who were 
1 selected to be in the class as language models for "their 
/ peers . 
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.q . TKe Bilingual Education Curriculum Program (B(|cP) was 

core set of instructional materials. However, teacher^ .^^ 
were allowed to supplement their lessons drawing materlals^ 
and' activities ffom other sources. 

o./Each class had a bidingual aide helping the teacher with 
instruction and class management. 




Parent al. involvement activities consisted of having parents 
attend workshops where parenting strategies were 'discussed. '[^^ 
Also, eacia unit of instruction had a home activity . 'H^^^-f 



HOW^ WERE THE PARTICIPANT^^S OF THE PROJECT SELECTED? 

The participants of the Project were randomly selected from a pool of 
applicants who were identified as LEP. In the AISD students are 
considered LEP if the following conditions were given: 

• . ' V ■ , • 

o A language other than English (for the purpose bf this 
project Spanish) is reported as~ being spoken at home, and. 

o -The student scores below criterion on the Primary 
Acquisition of Language Test (PAL). 

A random selection of students was used , to give jeach LEP applicant ah 
equal opportunity of being selected. 

The participating model students were selected from the pool of applicants 
^to the Project who obtained the highest scores on the PAL test, English 
version. Theore were three model students in each class of the Project. 

DID THE BILINGUAL PRESCHOOL PROJECT MEET ITS ACADEMIC ACHIEVEMENT OBJECTIVES? 

Project participants were tested twice during the school year with the 
PPVT-R. The. following finding were obtained from the results of testing: 

o The Bilingual Preschool Project had an impact on the 
■ English language skills of. the participants as demonstrated 
by th« gain obtained on the PPVT-R. Pre- and posttested 
LEP students obtained a net gain of 9.8 standard score 
. points in English. 

Furthermore, follow-up procedures indicate' that there was a lasting effect 
Data from the Iowa Tests of Basic Skills (ITBS) taken at kindergarten and 
first grade demonstrated that the former participants of the Project 
obtained higher scores than LEP . students who did not participate in 
the: District 's preschool programs. 



Figure 1 shows the results' of jthe -pre- and pqsttest for two of the classe 
of the Project- and the Pro j,ect " overall (82-83 school year)'. 



GROUP. 



PRETEST 
AVERAGE ■ 
SCORES- . 
(SD) . 



POSTTEST 
AVERAGE 
SCORES 
(SD) 



, AVERAGE 
•GAIN 

.(SD) 



t-value I^PROB. 

alpha 



PROJECT 
n=85 



73.36 
(21.26)' 



81.35 
(17.72) 



7.98 
(13.11), 



5.62 



.05 



claSs a* 

n=12 



54.25 
(26.70) 



, 74.25 
" (20.30) 



.20.5 ', 
(18.81).' 



3.77 



.003 



nHl 1 



66.90 
(19.61) 



66>.36 
(19.96) 



-.55 
(10.59) 



.17 



.86 



* Includes only LEP students with pre-^ and posttes.t valid scores, 



FIGURE 1. PPVT-R STANDARD SCORES GAINS FOR THE BILINGUAL PRESCHOOL 
• PROJECT AND FOR TWO CLASSES OF THE PROJECT )(82-83) . 



The results presented in 'Figure 1 indicate that class A obtained an 
average gain of 20.5 standard score points. This difference is 
statistically significant according to the. t-test conducted. 
However, the gain for class B was .55. The difference between pre- 
and posttest is not . statistically significant. 

Furthermore, an analysis of covariance was condacted to learn whether 
the class average of the posttests were different when adjusted for 
differences in pretest. It was found that there was a signifipant 
difference between posttest averages of the two classes compared. 
The results are presented in Figure 2. , . _ 



5 



SOURCE OF VARIATION 
COVARIATES 

* PRE 
MAIN effects: 

CLASS 
EXPLAINED • 
RESIDUAL' 
TOTAL 

1 : Sum of Squares 



SS 

3873.0 

1468.2 
5341.3 
3583.2 
8924.4 



DF 

1 

2 
20 
22 



MS- 



3873.0 21.61 • 



1468.3 
2670.6 
179.2 
405.7 



■ 8.19 
14.91 



PROB 

.001 

/ 

.010 

.oSi 



2: Degrees of Freedom 3: Mean Square' 



FIGURE 2.' ANALYSIS OF COVARIANCE OF PPVT POSTTEST AVERAGES BETWEEN TWO 
SELECTED CLASSES OF THE' BILINGUAL PRESCHOOL gROJECT. 

WAS ONE CLASS OJ THE PROJECT MORE EFFECTIVE THAN THE OTHER aASS COMPARED? 

Defining' effectiveness operationally as the attainment of high adjusted 
scores in the results of the PPVT-R, it can be stated that class A was 
more' effective than Qlass b; Therefore, it can be concluded that a 
greater development of language skills and greater academic achievement 
took place in class A than class B. For the purposes -of this study, 
effectiveness is 'defined only in terms of* results of one test. , We are 
not considering other effects that could have happened in the classes 
compared such as the development of greater self-esteem, or learning to 
be more independent. 

To understand what factors may have led to greater effectiveness, we 
compared the data gathered from the two classes with the following 
research techniques: 

o Structured Classroom Observations, and 

ft 

o Teacher Interviews. 

The description of the instruments used, the procedures followed, and 
' the findings obtained follow. 
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•A. Structured Classroom Observations ^ ' . 

The main purpose of the structured classroom observations was to document 
the objectives of the Project concerning .time (dedicated to instruction and ^ 
language used to administer instruction. The structured classroom obser- 
vations focused on th.e experience of^ a particular student during the 390 
minutes of the school day. A bilingual observer recorded information on 
a form developed by the .Office of Resiarch and Evaluation of the AISD 
for this specific prqject. The unit of observation of the form was. a 
school-day minute. The selected student was observed during an entire 
^day and . minute-by-minute records on several variables were taken of 
his/her personal experience. A copy of the f orm^ is included in Attachment 
A. . ' 

Classroom observations were cbnducted from March .18 to April 27, 1983, 
the sixth and seventh month of the school year for .this Project. 
We obseirved each Project ' s class three times\ Teachers did not 
know when their classes were going to tie observed. 

..Because the language . skills^ of the participants were heterogene*8^, the 
selection of the students to be observed^ was based' on language leveii. 
For each,, of ' the' observations irj, each school a' studenC. f rom a different; 
level was observed. A language level 'classification was. used^. The . 
categories were .thfe following : ^ ° . 

A students: Very low English p?roficiency as demonstrated by 'a range of 
scores on the PAL from 1 to 30. Spanish, monolinguals were 
included in this category. 

«' ' >^ « ■ . " » 

'B students:* Low English proficiency as demonstrated by a.- range of scores 
' - . on the PAL from 31 to 69 " ' ^ ' ^ 

C students: Average, English^ prof iciency and above as demonstrated by a 
range of scores on the PAL of 70 or above. / ' 

The observer selected avstudent from each, category to be observed at random, 
after the teaqher^ had, introduced all the children of the clas^. The 
teacher did not know wttich student was observed. 

For the purposes of this study the following variables were considered: 

o' Type of Instructional. Activities , and 

o Language of Instruction. 
Each .variable considered is discussed below. 
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A.l TYPES 6F INS T-RUCTIONAL ACTIVITY 

During each minute of the school day, ,the^ activities that took pl^ce were 
recorded into one of the following categories:. * ^ 

o Formal Learning, ' ' ~ 

o Informal Learning, 

- *o Independent Learning, and 

o Other. " ' . . 

The definition of each category was the following: 

Formal Learning: Activitie3 aimed at the presentation of new concepts, 

ideas, new' words, and the development of visual and 
auditory ^skills were recorded as formal learning. In 
the Project these activities took place in groups of ' 
less than six students. ^ 

Informal Learning: 'Activities whose objective was the reinfivrcement of 
A knowledge already learned during formal learning and 

other acti^ties. such as shating thoughts and/or 
' ' _ anecdotes »with the teacher, singing songs, listening- 
to stories, dancing, and playing group games. were 
coded as infonpal learning. 

Independent Learning: The period of 'time when a student is^allowed to be 

in a learning center working in an activity such as 
painting ,^t an easel or wrking with blocks was recorded 
as independent leading. It is acknowledged that the 
activities. that took place in these centers allowed 
students 'to explore educatipnal materials, expre*ss their 
creativity, learn social interaction skills, and also 
improve on their -language skills. *The activities at 
the learning centers took place in groups of less than 
four students. Because there was ;little interaction 
r with adults it is referred to as indepen^ient . 

Other:^ The period of time used by the , students for activities 

ot^her than the ones mentioned abgv=e"^s classified as 

)ther." They include meals, naps, recess, transitions 
f:^m-one activity to the next, bathxpom visits, and idle 
tile. . : 
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Findings Associated with Time Dedicated To Instructional Categories 



Figure 3 shows a comparison of the number of m^jmtSfis^ecorded for each 
of the instructional categories defined abg^e for each Df the classes 
obse'rved. • 




CLASS OBSERVED 



A-1 



FOPMAL 
LEAENING 



38 



INFO 
LEARN 



117 




INDEPENDENT 
LEARNING, 

57 



OTHER TOTAL 



178 



3 90 



A-2 • 


US 


38 


43 


1 94 


390 


A-3 


116 


34 


42 


198 


3 90 


TOTAL FOR CLASS 


269 


• 

189 

\ . 
63 

(46.8) 


142 


570 


1170 


AVERAGE ' 
(SD) 


89.6 
(44.7) 


47.3 
(8.4) 


190 
(10.5) 


3 90 


PERCENTAGE OF TPTAL 


. 22.9% 

' < 


16*. 2% 

\ 


12.1% 


48.7% 


'100% 


B-1' 


37 


58 


71 


224 


390 


B-2 


44 


64 


87 


195 


3 90 


Br-3 


23 


87 


77 


203 


3 90 


TOTAL FOR CLASS 


104 


209 


23 5 


622 


1170 ■ - 


AVERAGE 
(SD) 


34.7 
(10.7) 


69.7 
(15.3) 


( ^.1) 


207.3 
(14.9) 


390 


PERCENTAGE OF TOTAL 


8.8% 


17.9 


20.1 


53.1% 


100% 



FIGURE 3. COUNT OF MBJUTES RECORDED FOR EACH INSTRUCTIONAL CATEGORY. 
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The data presented in Figure 3 indicates that a major difference in the ' 
use of the school-day time between the classes compared was the time 
dedicated to formal leajffit^g and independent learning. Clas% A dedicated 
on-the average 23% (90 of 390 min.) of the time to formal learning and 
12% C47 of '390 min.) to independent learning. Class'B, instead, dedicated 
on*the average 9% (35 of 390 min.) to formal learning and 20% (78 of 390 min.) 
to independent learning. v ^ 

A. 2 LANGUAGE OF INSTRUCTION ' 

Ljanguage of instruction was another variable recorded during the structured 
classroom observations. During each minute of instruction observed the 
language heard by the' student was recorded. 

Findings Associated With Languaj^e of Instruction 

Figure 4 shows the number of minutes recorded for each^language during 
the periods of instruction for each class observed. The data collected 
during periods of formial and informal learning were aggregated. . 



CLASS 


ENGLISH 


SPANISH 


BOTH 


bla:;K ■ 


TOTAL 




136 


19 




•0 


155 


A-2 


* 96 


57 


0 


0 


153 


.A-3 


- ' 98 


40 


11 


1 


150 


TOTAL - 


330 


li6 


11- 


1 • 


453 


AVERAGE 
(SD) 


110 
C 22.5) 


33.6 
( 19.0) 


3.7 
( 6.4) 


0.3 
( 0.5) 


152.6 
( 2.5) 


Z OF CLASS 


" 72 .'ir: 


- 25.2% 


■ 2.4S 


0.22 


100% 


B-1 


'. 9^1 


0 


0 


4 


95 


a-2 , 


63 


36 


9 


0 


108 


B-3 • 


90 


20 


0 


0 


110 


TOTAL 


244 


56 


9 


4 


313 


AVERAGE 
(SD) 


"81.3 
( 15.9) 


18.6 
( 18.0) 


3.0 

( 5.2) 


1.3 

( 2.3) 


104.3 
( 8.1) 


' OF CLASS 


77.9% 


17.8% 


2.8% 


- 1.2% 


100% . 



.1: The observer heard« two languages during the observed minute • 
■2: TLe observer did not hear any languages spoken during the minute, 

^ *. .» 

^ FIGURE 4. • COUNT OF MINUTES DURING FOR-^IAL AND INFORMAL LSARNING BY 

LANGUAGE OF INSTRUCTION, |q 



r 



The data presented in Figure 4 yielded the following findings: 

o There was a small difference in the "proportion 

of time dedicated to instruction by language ^ 
and school. 

o The main difference, however, between the classes is 

not in the proportion of time dedicated to instruction 
. by language but in the. amount of time dedicated to 
instruction overall. 



B. Teacher Interview 

' ' — 7 — ^ 

Interviews were conducted with each of the teachers of the AISD 
Bilingual Preschool Project. The purpose of the interviews was 
to address the following research questions: ' 

o What goals were set by each teacher for their classes? 

o What set of instructional materials were used? 

o What criteria were used by teachers to form groups 
^ for instruction? 

o What decisions were 'made with respect to language 
^ of instruction? 

The interviews conducted followed a preastablished format. The questions 
'included in this format were related to the objectives of the Project andv 
other descriptive information. The questions ask^d were open-ended. 
Topics were followed up when, an illustration or more information was 
needed. - • 



Interviews were conducted vith each teacher during the last week of 
classes. May 23 to May 27, 1983. The interviews took place at each 
school. Teachers were assured that coimnents shared during the interview 
would not affect whatsoever theit professional evaluation and that all 

the information collected would remain anonymous. 

^ * - ■ 

^^The results of the interviews with the teachers of the classe^ compared 
in t-his paper are presented according to 'the topics addressed^ 
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FINDINGS 

B.l Teacher Goals 

I' • . . . ■ ■ • ^ ■ "Jr. 

Teachers were asked .toHdentify the goals they set for their ^classes. 
It was assumed that the ^sfe^feed igoals 'indicate the ar'feas^in which 
teachers would concentrate thei^^activities and efforts. We found . 
differences in the goals set 1)y \he teachers. .The teacher from class 
A had goals related almost; e'xclusVyely to developing language skills.- 
The other teacher, however,. -611 p ha sis ed develbpiijg the affective ^ 
area of the students. The f o'll6wing\resporifees"were given by each 
teacher: 

Goals set by teacher A: . ^ 

1. "For the Spanish dominant students to- learn English." 

2. "Foi?M:he English dominant students to learn Sl^anish/\ 
3i "To improve overall language skills/' . 



Goal^ set by teacher B: ^ 

1. "To teach students to communicatee with each other." ^ 

2. "To teach students to get -along and see each oth^r as an 

important individual.'" • ' " . . 

3. "To get students ready for next gr^da^evel"^ so they would 
•not feel los^t." ' , " 

. ■ ■ . .":r7' ^ . . . — . 

:B.2 Set of Instrucfciohal Materials 

All teachers df^theJPriiject used the Bilingual Education Curriculum • 
Program (BECP) , a set of instructional materials developed by the 
Southwest Educational Development Laboratory. However, teachers 
were allowed to use other materials to supplement instructj-ion. 
It was found 'during^ thB previous years of the Project that this set 
of materials was adequate for students with vdry low. English skills ^ 
and Spanish monolinguals but it was not challenging to^tuden1:s at a 
higher level of language development. . 

Differences between the two teachers of the study were noted ^once again 
Teacher B used it 80% of the time. while tea^ch^r A reported using it ^ 
only 35% of the time. These percentages .were' estimates given by the ^ 
teachers. . \ " . 

Anoth^er difference between these two classes, thit ei&erged from the 
interviews was the use of a book of stories which consists of 
mathematical problems and high level thinking. Teacher. A read ^stories • 
'from this book,. Real Math 1, twice a week since February, t^^. The teaciher 
shared these comments: . ■« • . ' . ^ 
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"The stories from Real Math exposed the students to math 
concepts and high level thinking. The majority of the 
Jt • students were able to follow the stories. My emphasis 

was on the stories and there was a secondary interest 
on the math concepts elaborated. About 50% of the 
/Students didvnot grasp^ the' math concept, but all 
gained oral language development and thinking skills. 
After I told the story, I explained the conclusion 
and students after an explanation understood the answers 
or solutions given." 

Because the teacher that used this type of instructional materials 
obtained higher than average achievement gains, it is appropriate 
to raise the following research question: 

o Does the use of high level thinking instructional 
materials lead to a higher development of language 
skills among limited English proficient students? 

^ 

B.3 Groups for Instruction 

All teachers of the Project reported using small groups (4-6 students 
per group) for instruction. The most common criterion used to form 
the small group by teachers was language level. Students assigned 
each student to a group based on their own observations and information 
provided by the results of the PAL testing. 

We asked the teachers whether they have changed the groups through 
the year. There were differences in the responses given by the 
teachers compared. Teacher A reported moving the children from 
one group to another when, in her judgemetjit and observation, the 
language level of the students changed. The other teacher kept 
the groups stable through the year with few changes and those based 
upon discipline problems. 

B.4 Language Question 

In bilingual education projects where language heterogeneity among the 
students is a common condition, instructors are faced with the question 
of what language to use during periods of instruction with the entire 
c^ass. The alternatives have been to "stay clean" or to "flip-flop." 
The former term refers to conducting the entire session of instruction 
in only one language, though some of the students may be lost. Both 
teachers compared in this study reported using only one language during 
the period of instruction and translating briefly to those who did 
not understand at the end of the period. 
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SUMMARY AND RECOMMENDATIONS ^ 

Two classes with ^reat differences in academic achievement were compaced 
to identify factors that might account for differences in gains. 

The following recommendations are presented as hypothesis to be tested 
that may lead to Project enhancement: 

1. Increase the Gime dedicated to formal and informal 
learning. 

2. Reduce the time students spent working in indefiendenD 
learning centers^. 

3. Use two languages for (instruction but English should be 
^ . ' ^--^^he predominant langu^e of the class, 

> 

4. Use a variety of instructional materials that are adequate 
to the language level of the students, including high 
level thinking materials. 

5. Rotate students to new groups for instruction when the 
language level of the students change. 

6. Conduct the enti;re instructional period in one 
language and then translate briefly to students 
who do not understand. 

7. Sejt language development as the main goal of the 
class for the school year. 
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Lang. Group: 



*N0TE: Independent learning was 
cod ed . und er - iiif ormal • >_ 
learning. However, different 
•codes were used ( "1" for 
informal learning per se 
and a "2" for independent 
learning 
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SIDE 3 

OBSERVATION NO. 
Name of Student 
School : 
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Observer: 
Date: 



Lang. Group; 
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